Image detection and processing for building control

ABSTRACT

Occupancy detection for building environmental control is disclosed. One apparatus includes at least one image sensor and a controller, wherein the controller is operative to obtain a plurality of images from the at least one image sensor, determine a color difference image of a color difference between consecutive images of the plurality of images, determining an area of the color difference image wherein the color difference is greater than a threshold, create an occupancy candidate based on filtering of the determined area if the determined area is less than light change threshold, generate one or more centroids that represent areas of the occupancy candidate that are greater than a second threshold, track the one or more centroids over a period of time, detect occupancy if one or more of the centroids is detected to have moved more than a movement threshold over the period of time.

RELATED APPLICATIONS

This patent application claims priority to U.S. Provisional PatentApplication Ser. No. 61/699,946, filed Sep. 12, 2013, which is hereinincorporated by reference.

FIELD OF THE EMBODIMENTS

The described embodiments relate generally to building controls. Moreparticularly, the described embodiments relate to image detection andprocessing for building control.

BACKGROUND

Lighting control systems automate the operation of lighting within abuilding or residence based upon, for example, preset time schedulesand/or occupancy and/or daylight sensing. The Lighting systems typicallyemploy occupancy sensors and/or daylight sensors to determine whichlighting devices to activate, deactivate, or adjust the light level of,and when to do so. Occupancy sensors typically sense the presence of oneor more persons within a defined area and generate signals indicative ofthat presence. Daylight sensors typically sense the amount of daylightpresent within a defined area and generate signals indicative of thatamount. Typically, lighting systems receive the sensor signals at acentral lighting controller.

The lighting systems are advantageous because they typically reduceenergy costs by automatically lowering light levels or turning offdevices and appliances when not needed, and they can allow all devicesin the system to be controlled from one location.

It is desirable to have a method, system and apparatus for improvedmotion and occupancy detection.

SUMMARY

One embodiment includes an apparatus. The apparatus includes at leastone image sensor and a controller, wherein the at least one image sensoris associated with an area of interest, which may exist (for example)within a structure or building. At least one of the controller, or anexternal controller electronically connected to the controller areoperative to obtain a plurality of images from the at least one imagesensor, determine a color difference image of a color difference betweenconsecutive images of the plurality of images, determine an area of thecolor difference image wherein the color difference is greater than athreshold, create an occupancy candidate based on filtering of thedetermined area if the determined area is less than light changethreshold, generate one or more centroids that represent areas of theoccupancy candidate that are greater than a second threshold, track theone or more centroids over a period of time, detect occupancy if one ormore of the centroids is detected to have moved more than a movementthreshold over the period of time, and control at least oneenvironmental parameter of the structure based on the detectedoccupancy.

Another embodiment includes a method of detecting occupancy. The methodincludes obtaining a plurality of images, determining a color differenceimage of a color difference between consecutive images of the pluralityof images, determining an area of the color difference image wherein thecolor difference is greater than a threshold, creating an occupancycandidate based on filtering of the determined area if the determinedarea is less than light change threshold, generating one or morecentroids that represent areas of the occupancy candidate that aregreater than a second threshold, tracking the one or more centroids overa period of time, and detecting occupancy if one or more of thecentroids is detected to have moved more than a movement threshold overthe period of time.

Another embodiment includes another method of detecting occupancy. Themethod includes obtaining a plurality of images, determining a colordifference image of a color difference between consecutive images,determining areas of the color difference image wherein the colordifference is greater than a threshold, calculating a total change areaas an aggregate area of the determined areas, creating a list ofoccupancy candidate regions based on filtering of the determined areasif the total change area is less than a light change threshold, whereineach occupancy candidate region is represented by a connected component,tracking the one or more occupancy candidate regions over a period oftime, and detecting occupancy if one or more of the occupancy candidateregions is detected to have moved more than a movement threshold overthe period of time.

Other aspects and advantages of the described embodiments will becomeapparent from the following detailed description, taken in conjunctionwith the accompanying drawings, illustrating by way of example theprinciples of the described embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a building that includes an image sensing building controlsystem, according to an embodiment.

FIG. 2 shows a building that includes an image sensing building controlsystem, according to another embodiment.

FIG. 3 shows a lighting control sub-system, according to an embodiment.

FIG. 4 shows a smart sensor system and an IR Illuminator, according toan embodiment.

FIG. 5 is a flow chart that includes steps of a method of detectingoccupancy, according to an embodiment.

FIG. 6 shows a lighting control sub-system, according to anotherembodiment.

FIG. 7 shows a room wherein the reflectivity of objects within the roomis characterized, according to an embodiment.

FIG. 8 is a flow chart that includes steps of another method ofdetecting occupancy, according to an embodiment.

DETAILED DESCRIPTION

The described embodiments include image sensing and sensed imageprocessing for controlling environmental parameters of a building,structures, or an area of interest. For at least some embodiments, theenvironmental parameters include lighting, heating, cooling, humidity,air flow and others. Based on image sensing and processing, any of theenvironmental parameters can be controlled. While the describedembodiments are primarily focused on environmental controls, it is to beunderstood that other embodiments include other types of buildingcontrols, such as, building security.

FIG. 1 shows a building that includes an image sensing control system,according to an embodiment. While FIG. 1 represents a building, it is tobe understood that the described embodiments for controlling lightand/or an environmental parameter(s) can be utilized in any type ofstructure. At least one image sensor of the image sensing control systemis arranged to capture images of an area of interest 100. The area ofinterest can include, for example, a building, a structure, a parkinglot, and/or a garage. Additionally, the area of interest can be adifferent area than where the environmental controls are being applied.The system includes at least one image sensor, such as, image sensors110, 112. The image sensors 110, 112 sense images within the area ofinterest 100. For at least some embodiments, a server 120 (and for someembodiments, other controller as well) receives the sensed images,performs processing on the images, and then provides environmentalcontrol of the area of interest 100, or any other area, based on theimage processing of the sensed images. Exemplary building and/orstructure (such as, a parking lot) environmental and/or securitycontrols include lighting control 160 and/or HVAC control 170. Clearly,additional or alternative building control (such as, security) can beincluded. Additionally or alternatively, building analytics canmonitored.

The image sensors 110, 112 include any type of sensor that generates asequence of images (continuous, triggered or other), such as, a videocamera.

For at least some embodiments, the server 120 may perform processing onthe sensed images immediately after they are captured by the imagesensors 110, 112. In such embodiments, the image processing is performedin an incremental manner, resulting in environmental control outputsthat are available with a minimal time delay after any triggering eventtakes place in the area of interest. For other embodiments, theprocessing may take place in a batch mode, where a number of images aregathered from an image sensor and then processed together. While severalof the described embodiments may suggest pipeline processing, it is tobe understood that the described embodiments for image processing can beimplemented according to pipeline processing, batch processing or acombination of both pipeline processing and batch processing.

For at least some embodiments, the image sensors 110, 112 are utilizedto sense occupancy by users 150 within the area of interest 100. Theenvironmental control is based at least in part upon the occupancy ofthe users. For at least some embodiments, the environmental controlprovides a more comfortable environment for the users 105. For at leastsome embodiments, the environmental control provides a more costefficient and/or cost effective use of the area of interest.

As shown, the area of interest 100 includes conditions that can inducefalse detection of human presence within the area of interest. Forexample, a blinking LED on an electronic device 130 can provide a falsedetection of presence by the image sensors 110, 112 and the imageprocessing. Further, mechanical motion of, for example, the paperassociated with a printer 140 can provide a false detection of presenceby the image sensors 110, 112 and the image processing. Further, shadowsand reflection passing through a window 150 of the area of interest 100can provide a false detection of presence by the image sensors 110, 112and the image processing. The area of interest 100 may be locatedadjacent to, for example, an open area 190 that may include lighting(such as, light fixture 192) that emits extraneous light (cross-talklight) that is at least partially received by the image sensors 110,112, which can also provide false detection of presence by the imagesensors 110, 112 and the image processing. Accordingly, the imageprocessing should take into account the possible conditions within thearea of interest 100 that can cause false positives of occupancydetection, thereby providing improved environmental control andanalytics monitoring of the area of interest 100.

For simplicity, camera-based (image-sensor based) occupancy detectionsystems are described based around the idea that any occupancy in ascene will intermittently be associated with motion and that the motioninduces local changes in the image which can be detected by simple framedifferencing. However, detecting occupancy in the scene by taking framedifferences should properly recognize ambient lighting change events andreject them as not being motion events. It should be noted that cameraautomatic gain adjustments can cause similar artefacts. Similarly,detecting motion in the scene by taking frame differences needs to beable recognize that other image changes that are not associated with anyoccupancy, such as flashing LEDs should be ignored. These artefactsshould be correctly accounted for by the system and system processing,as will be described.

A lighting change or camera automatic gain adjustment typically does notinvolve any motion in the scene. Generally, the region of the image thatchanges is stationary. As far as the frame difference processing isconcerned the main thing that distinguishes a lighting change event froman occupancy candidate region is that it causes a frame difference overa large contiguous area in the scene whereas frame differences arisingfrom humans moving are relatively small and fragmentary.

At least some of the described embodiments that include tracking allowfor the detection of ‘non-motion’. For example, a blinking LED or ashiny surface in the scene that picks up occasional reflections mayinduce a frame difference which exceeds the frame difference threshold.At least some of the described embodiments recognize that there is nomotion associated with this category of event. By recording the initiallocation of a frame difference event and keeping track of the maximumdistance that the motion point moves from its initial location, at leastsome of the described embodiments are able to recognize these‘non-motion’ events as being ones that do not move from their initiallocation.

As previously stated, for at least some embodiments, the server 120receives the sensed images, performs processing on the images, and thenprovides environmental control of the area of interest 100 based on theimage processing of the sensed images.

FIG. 2 shows a building that includes an image sensing building controlsystem, according to another embodiment. This embodiment includes smartimage sensors 210, 212 that include image processing being performed bya controller of one or more of the smart image sensors 210, 212. Forvarious embodiments, the image processing can occur at a single one ofthe controllers (such as, a controller of a smart image sensor, anexternal controller interfaced with the image sensor(s), or a completelyexternal controller in the cloud) or the image processing can bedistributed amongst multiple of the controllers.

Whether the image processing occurs at a single controller or isdistributed amongst a plurality of controllers, at least someembodiments include image processing wherein at least one of thecontrollers is operative to obtain a plurality of images from the atleast one image sensor, determine a color difference image of a colordifference between consecutive images of the plurality of images,determining an area of the color difference image wherein the colordifference is greater than a threshold, create an occupancy candidatebased on filtering of the determined area if the determined area is lessthan light change threshold, generate one or more centroids thatrepresent areas of the occupancy candidate that are greater than asecond threshold, track the one or more centroids over a period of time,detect occupancy if one or more of the centroids is detected to havemoved more than a movement threshold over the period of time, andcontrol at least one environmental parameter of the area of interest, ora different area of interest, based on the detected occupancy.

Determining a Color Difference Image of a Color Difference BetweenConsecutive Images of the Plurality of Images

For at least some embodiments, if a gray scale is used in thedetermination of the color difference image of a color differencebetween consecutive images of the plurality of images, the grey scaleinformation may be obtained from a camera's luminance channel, or byconverting RGB values to a grey value, or by extracting a single colorchannel from the camera's output, for example the green channel. Thedifference between two consecutive grey scale images is obtained fromthe absolute difference in the grey scale values.

If color differences are to be used, the difference between twoconsecutive images can be computed using, for example, one of the CIEDelta E measures of color difference, such as, CIE76 CIE94 or CIEDE2000.Computation of these color differences measures require that the inputimages are converted to the CIE L*a*b* or CIE L*C*h color space.

For at least some embodiments, an image frame is obtained from avideo/image stream and down-sampled to a suitable size. For anembodiment, the image frame has a size of 60×100, but clearly differentsizes are possible and in fact preferred. An embodiment includesconverting to greyscale. An embodiment includes sampling a green channel(or other selected channel) rather than performing a full greyscaleconversion. An embodiment includes using a luminance output of a camerato represent the greyscale.

Determining an Area of the Color Difference Image Wherein the ColorDifference is Greater than a Threshold

For an embodiment, the absolute value of the difference between apresent frame and the previous one is computed and threshold detected atsome level to eliminate noise. Any differences that exceed the thresholdlevel are assumed to be the result of something moving in the intervalbetween the frames. In such an embodiment, the difference between twosuccessive frames caused by a moving object typically produces an‘outline’difference images because only the edges of the moving objectproduces a difference relative to the previous frame.

For an embodiment, the plurality of images represents compressed imagedata from a video or still camera. This compression introducesquantization of the image values, and other artefacts, which can beconsidered forms of noise. Also, the changing of the keyframes that areused in some forms of compressed video introduces additional noise whichappears in the difference image intermittently. The difference thresholdshould be large enough so that compression artefacts are ignored.

An embodiment includes a simple recursive IIR temporal filter as apreprocessing step on the input images in the processing. This is usefulin enhancing the sensitivity of the processing to small motions.

For an embodiment, a simple recursive IIR filter is arranged so as toform a ‘background image’ against which the current image is compared,for the purpose of finding image differences. The IIR filter is a simpleway by which a background image can be obtained without being undulyaffected by people or other moving objects in the scene. After filteringstationary objects in the scene will appear in the image with low noiseand thus it is possible to use a lower, more sensitive, image differencethreshold.

For an embodiment, the difference between the present frame and theIIR-filtered background image is then computed and threshold-detected.An object moving in the scene will produce a solid difference regionbecause all parts of the object will be different to the background. Ifa light change event is detected then the reference background image isreset to the current (post light change) image and the IIR temporalfiltering restarted.

Described more generally, the idea is to maintain a ‘background image’against which the current image is compared against for the purpose offinding image differences. The IIR filter is a simple way by which abackground image can be obtained without being unduly affected by peoplemoving through the scene.

The main advantage of this is that an object moving in the sceneproduces a ‘solid’ difference region because all parts of the objectwill be different than the background. If one simply takes thedifference between two successive frames a moving object will typicallyproduce an ‘outline’ difference images because only the edges of themoving object will produce a difference relative to the previous frame.

Note that this only works well if we do not have any lighting changes.If a light change event is detected then the reference background imageneeds to be reset to the current (post light change) image and the IIRtemporal filtering restarted.

Creating an Occupancy Candidate Based on Filtering of the DeterminedArea if the Determined Area is Less than Light Change Threshold

For at least some embodiments, if the proportion of the area of imagethat has changed is greater than some threshold, it is assumed this hasbeen caused by a lighting change; this is termed a light change event.For an embodiment, a ‘lightChange’ flag is set for a predeterminedperiod of time after a light change event. While the lightChange flag isset, difference images are not calculated. The idea of having a‘lightChange’ flag is to allow any transients, gain control changes etc.to settle down before resuming normal operation of the system.

In some situations, the lighting change can be detected one frame toolate. Transients leading into the lighting change then input to theprocessing as artefacts which introduce noise and possible incorrectlyidentified connected components into the processing. An embodimentaddresses this by storing a temporal buffer of input frames. Generationof a difference image on a delayed pair of frames is predicated on thelight change status of the current frame pair. If there is no lightchange, the delayed frame pair is processed as usual, updating thesystem state. If there is a light change in the current frame pair, thedifferencing of the delayed frame pair is abandoned, and the systemstate remains unchanged. This avoids the processing being affected byleading transients to a lighting change.

If a lighting change has been deemed to have occurred there is nothingthe processing can sensibly compute from the image. The processing thenstarts over, and waits for the next frame, again testing the total areaof the image that changes in the next frame difference.

For an embodiment, if a lighting change has not occurred, the differenceimage is then filtered to determine regions of motion. The differenceimage resulting from a moving object will typically consist of a numberof disjoint regions. These are merged together to form an occupancycandidate region. In addition there may also be spurious differenceregions resulting from noise in the images which need to be eliminated.

The difference image is filtered with a Gaussian, or Gaussian-likefilter. This blurs and smoothes the difference image. The filteredresponse is strongest where the density of difference regions is high.Thesholding this smoothed image at an appropriate level has the resultof coalescing the disjoint regions in the difference image produced by amoving object into a single connected component (or at least a smallnumber of connected components). The thresholding also eliminates smallspurious difference regions arising from noise because these occur at alow density and thus do not produce a strong filtered response.

For an embodiment, Gaussian-like filtering is achieved efficiently byapplying a summation filter recursively to its own output three timesover. This allows filtering to be performed in time proportional to thenumber of image pixels independent of the desired filter size and onlyrequires integer arithmetic. In another embodiment, an averaging boxfilter is recursively applied to its own output three times over toobtain an equivalent result. A summation filter is equivalent to anaveraging box filter but without the division by the number of elementswithin the box filter being applied.

An embodiment of the summation filter includes an averaging filter butwithout the final division by the number of pixels being performed. Thismeans that integer arithmetic can be used. For efficiency, the describedembodiments exploit the separability of the filter and perform a 1Drunning summation on the rows of the image followed by a 1D runningsummation of the columns of the resulting image to achieve the filteredresult. This runs in time proportional to the number of pixels in theimage and is independent of the filter size.

The use of a Gaussian or Gaussian-like filter means that pixels in thedifference regions close to the center of the filter have more weight,so outlying pixels of the difference regions do not shift thethresholded result around in pixel space, leading to much more stableoccupancy candidate region positions and more reliable occupancycandidate region tracking

For at least some embodiments, the filtered image is then thresholded ata low value, which defines the regions of motion. The thresholdingresults in a binary image consisting of groupings of connected pixels,known as connected components, with each connected componentrepresenting a occupancy candidate region.

For an embodiment, a labeled version of the motion region image isgenerated. That is, the pixels of each separate ‘connected component’ inthe motion region image are given a unique label.

Generating One or More Centroids that Represent Areas of the OccupancyCandidate That are Greater than a Second Threshold

For an embodiment, centroids of each labeled connected component arecomputed and passed to the tracking code. The motion of the centroid isassumed to represent the motion of the object.

Tracking the One or More Centroids Over a Period of Time

For an embodiment, the tracking code maintains a list of connectedcomponent centroids. For an embodiment, for each centroid it maintainsthe following: the initial position of the centroid, the currentposition of the centroid, the maximum distance the centroid has movedfrom its initial position and the time since the centroid was lastupdated.

When the tracking code is called with the current list of connectedcomponent centroids, the tracking code attempts to match up the currentcentroid positions with the centroids in the current list it ismaintaining For those that are matched up the current position andmaximum distance from initial positions is updated. Centroids in thelist that were not matched have their time since centroid was lastupdated incremented. Any of the new centroids that were not matched toanything are added to the centroid list as a new centroid to be tracked.

Detecting Occupancy if One or More of the Centroids is Detected to haveMoved More than a Movement Threshold Over the Period of Time

Tracked centroids are classified as either being ‘non-moving’ or ‘hasmoved’. A ‘non-moving’ centroid is one that has never moved more thansome threshold distance from the initial location at which it wasdetected. The aim is to capture image frame difference connectedcomponents that result from things such as flashing LEDs, screen savers,oscillating fans, etc under this category.

When a connected component is first detected it is initially classifiedas non-moving. When its tracked motion takes it beyond a motion distancethreshold from the location it was first detected, its classification ischanged to has moved. Once a connected component has been classified ashas moved, it stays with that classification (Thus, has moved reallymeans this object has moved at some stage).

If a tracked connected component does not move for a certain number offrames it eventually dies and is removed from the list of connectedcomponents being tracked. The number of frames before a non-movingconnected component is deleted is less than that for a “has moved”connected component.

For an embodiment of lighting control logic, all the updated motioncentroid tracks are searched. If there are any tracks where there hasbeen motion and that track has been more than the motion distancethreshold from its initial position (that is, it has a classification of‘has moved’ and is thus not a flashing LED or similar), the timer isreset. Otherwise the light timeout timer is decremented. If the timerreaches 0, the lights are turned off.

The use of the motion connected component centroids as input to thetracking process keeps the number of objects that has to be tracked to aminimum. However there is a computational cost in performing the binaryimage labeling.

Processing Enhancements

At least some of the described embodiments provide discriminationbetween likely-humans and other connected components in the image.Connected components that move a significant distance are not a problem.For humans who are moving minimally (for example, sitting at a deskreading), any of the following described techniques can be used.

An object that is moving will typically appear in the raw differenceimage as an outline because it is mostly the moving edges of the objectthat create the frame differences. (The frame difference between thecenter of the object and a slightly moved center of the object willtypically not be as large). On the other hand a frame differencearising, say, from an object suddenly reflecting light from a newsource, or a computer screen waking up or going to sleep will produce asolid connected component in the raw difference image. The differencebetween the two forms of frame difference characteristics can berecognized by analyzing the topology of the difference image regionsand/or the topology of the connected components obtained after filteringand thresholding.

An object that has been tracked from the edge of the image inwards isgiven special status because this has a higher probability of being ahuman. So even if that connected component stops moving for aconsiderable length of time, as would be the case if the person sat downand was reading, that connected component can still be retained withinour list of candidate humans.

Over time each tracked connected component makes, or does not make, amovement. This movement forms a temporal sequence. The ‘rhythmicsignature’ of this sequence generated by a human is quite different tothat generated by an inanimate object. A classification scheme for thistemporal sequence is devised, using traditional signal processing ormachine learning techniques. The rhythmic signature of each candidateconnected component is then classified and this classificationconditions the list of candidate humans being tracked. A feature vectorof attributes of each connected component being tracked is formed toimprove the reliability of tracking and perhaps allow the recognition oftracked objects that have exited the area of interest and thenre-entered it. For an embodiment, the attribute of each connectedcomponent is its centroid.

Additional attributes can also be included. For example, the centroidvelocity can be included. A histogram of grey values or color values ofthe pixels in the input image that correspond to the pixels in theconnected component can be included. A histogram of gradients of greyvalues of the pixels in the input image that correspond to the pixels inthe connected component can be included.

Clustering of connected components can be included. Despite thefiltering to coalesce regions of the difference image into largerconnected components it is still possible that some moving objects mayend up being represented by two or more connected components. These canbe merged by performing a clustering operation on the connectedcomponents on the basis of their proximity and on their attributevectors. For at least some embodiments, a feature vector of attributesis associated with each connected component and is stored, so as toimprove the reliability of tracking and allow the recognition of trackedobjects that have exited the area of interest and then re-entered it.The feature vector contains the connected component's centroid, and oneor more of the following parameters: the centroid velocity, a histogramof grey values or color values of the pixels in the current frame thatcorrespond to the pixels in the connected component, and/or a histogramof gradients of grey values of the pixels in the current frame thatcorrespond to the pixels in the connected component. Connectedcomponents are formed into clusters, to associate together multipleconnected components that are result from a single moving object. Insome cases, the filtering to coalesce regions of the difference imageinto larger connected components may result in certain moving objectsbeing represented by two or more connected components. These two or moreconnected components are merged by performing a clustering operation onthe connected components, on the basis of their proximity together withtheir feature attribute vectors.

FIG. 3 shows a lighting control sub-system 300, according to anembodiment. For at least some embodiments, the lighting controlsub-system 300 is operable as the image sensors 110, 112 of FIG. 1, oras the smart image sensors 210, 212 of FIG. 2. The lighting controlsub-system 300 includes many other sensors than just the image sensor.For at least some embodiments, the other sensor can be used tosupplement the motion and occupancy sensing of the described embodimentsfor motion and occupancy sensing through image sensing and processing.For an embodiment, the image sensing and processing is used to controlthe lighting intensity of a luminaire 310.

The exemplary lighting control sub-system 300 includes a smart sensorsystem 302 that is interfaced with a high-voltage manager 304, which isinterfaced with the luminaire 310. The high-voltage manager 304 includesa controller (manager CPU) 320 that is coupled to the luminaire 310, andto a smart sensor CPU 335 of the smart sensor system 302. As shown, thesmart sensor CPU 335 is coupled to a communication interface 330,wherein the communication interface 330 couples the controller to anexternal device. The smart sensor system 302 additionally includes asensor 340. As indicated, the sensor can include one or more of a lightsensor 341, a motion sensor 342, and temperature sensor 343, and camera344 and/or an air quality sensor 345. It is to be understood that thisis not an exhaustive list of sensors. That is additional or alternatesensors can be utilized for lighting and/or environmental control of anarea of interest that utilizes the lighting control sub-system 300. Thesensor 340 is coupled to the smart sensor CPU 335, and the sensor 340generates a sensed input.

According to at least some embodiments, the controllers (manager CPU 320and the smart sensor CPU) are operative to control a light output of theluminaire 310 based at least in part on the sensed input, andcommunicate at least one of state or sensed information to theluminaire.

For at least some embodiments, the high-voltage manager 304 receives thehigh-power voltage and generates power control for the luminaire 310,and generates a low-voltage supply for the smart sensor system 302. Assuggested, the high-voltage manager 304 and the smart sensor system 302interact to control a light output of the luminaire 310 based at leastin part on the sensed input, and communicate at least one of state orsensed information to the external device. The high-voltage manager 304and the smart sensor system 302 can also receive state or controlinformation from the external device, which can influence the control ofthe light output of the luminaire 310. While the manager CPU 320 of thehigh-voltage manager 304 and the smart sensor CPU 335 of the smartsensor system 302 are shown as separate controllers, it is to beunderstood that for at least some embodiments the two separatecontrollers (CPUs) 320, 335 can be implemented as single controller orCPU.

For at least some embodiments, the communication interface 330 providesa wireless link to external devices (for example, a central controllerand/or other lighting sub-systems).

An embodiment of the high-voltage manager 304 of the lighting controlsub-system 300 further includes an energy meter 322 (also referred to asa power monitoring unit), which receives the electrical power of thelighting control sub-system 300. The energy meter 322 measures andmonitors the power being dissipated by the lighting control sub-system300. For at least some embodiments, the monitoring of the dissipatedpower provides for precise monitoring of the dissipated power.Therefore, if the manager CPU 320 receives a demand response (typically,a request from a power company that is received during periods of highpower demands) from, for example, a power company, the manager CPU 320can determine how well the lighting control sub-system 300 is respondingto the received demand response. Additionally, or alternatively, themanager CPU 320 can provide indications of how much energy (power) isbeing used, or saved.

For the embodiments described, the lighting control system includes aplurality of the lighting control sub-system, such as the lightingcontrol sub-system 300 of FIG. 3. Generally, the purpose of the lightingcontrol system is to control illumination so that the appropriateillumination is provided only when and where it is needed within abuilding or area of interest in which the lighting system is located.Ideally, the lighting control system operates at the lowest cost overthe lifetime of the lighting control system. The costs include both theinstallation (generally referred to as the CAPEX (capital expenditure))and the ongoing operational cost (generally referred to as the OPEX(operational expenditure)), which includes the energy costs and themaintenance costs (such as bulb replacement, inspection), and operationof a central controller. The actual illumination needs generally changesover the lifetime of the lighting control system due to changingbuilding occupancy levels and usage, and by changes in occupant needs. Abuilding that is expected to see changes in illumination needs, requiresa lighting control system that provides low cost changes to reduce thetotal cost and to extend its lifetime. Additionally, as the technologyof luminaires changes over time, the ideal lighting control systemsupports replacement of the luminaires without the need to replacecomponents of the lighting control system. While lowering the costs, theideal lighting control system should enhance occupant productivity,well-being, and security. Also, the ideal system should gatheroperational status and statistics so that the actual operation can becompared with the desired operation, and the design assumptions (such asoccupancy patterns) can be validated, and the configuration changed, ifneeded, when the design is changed to match the actual use.

At least some embodiments of the lighting control system include aplurality of the lighting control sub-system. Each of the lightingcontrol sub-systems can operate independently, in coordination withother lighting control sub-systems (for example, existing hard-wiredsystems), and/or in coordination with a central controller. As such,each of the lighting control sub-systems can be independently installed,and adapt their operation accordingly.

As previously stated, an embodiment of the lighting control sub-systemincludes a communication interface, a controller (listed in discussionas a single controller, but as previously described, at least someembodiment include multiple controllers, such as, the high-voltagemanager 304 and the smart sensor CPU 335), a luminaire, a light sensor,and an image sensor. For an embodiment, the luminaire is a lighting unitconsisting of one or more lamps, socket(s) and parts that hold thelamp(s) in place and protect them, wiring that connects the lamp(s) to apower source, and reflector(s) to help direct and distribute the light.Various embodiments of luminaires include bulb technologies, such asincandescent, florescent, and LED (light emitting diode). Further,various embodiments of the luminaires are controllably turned on andoff, and further, are controllably dimmable.

For at least some embodiments, the controller makes decisions as toturning on, turning off, and dimming the luminaires. The controller doesthis, for example, either due to command from an external device (suchas, the central controller), or by processing decision rules usinginputs from the sensors, a saved configuration, time of day, passage oftime from past sensor inputs, and/or from state or sensor values fromother sub-systems. Additionally or alternatively, learned behavior caninfluence the decisions.

In its most basic configuration, the controller only controls turning atleast one luminaire on or off (no dimming) using simple rules and onlyinputs from an image sensor and light sensor and passage of time. For anembodiment, the controller is split into two physical modules as shownin FIG. 3. The first module is called the powerpack (referred to as thehigh-voltage manager 304), and contains the following sub-modules: powertransformer (AC to DC voltage), relay to disrupt power to the luminaire,power meter, one module sends dimming control signal (and other modelpasses a dimming control signal) to the luminaire, and “watch dog”support to restart the other module of the controller if it becomesunresponsive. The second module (referred to as the smart sensor system302) houses the image, and temperature sensors, a processor, persistentstorage for configuration, and the wireless interface. The processor inthis module executes the rules for controlling the light level of theluminaire.

For an embodiment, the controller is co-located with the luminaire.Also, at least some embodiments of the luminaire have chambers whereinthe controller can be housed, or are designed for an external chamber tobe attached that can house the controller.

For at least some embodiments, the controller intercepts power sourcesgoing to the luminaire and provides on/off controlled power. Anembodiment of the controller also provides a 0 to 10 v control signal tothe luminaire, and if supported by the luminaire, for dimming.

For at least some embodiments, the sensors sense (or measures) somephysical quantity and converts it into a digital value. For anembodiment, the sensors are packaged together with the controller. Morespecifically, for various embodiments of the lighting controlsub-system, multiple sensors of the lighting control sub-system includean image sensor, a light sensor, and temperature sensors located in thesame physical module, which is connected to the other physical moduleswith a cable. For an embodiment, the sensor(s) are physically locatedbeside the luminaire, and the image and light sensors are directedtowards the floor of an area of interest in which the lighting controlsub-system is located. For an embodiment, the sensor(s) are directlyconnected to the controller.

For an embodiment, the controller is further operative to receiveinformation from an external device, wherein the received informationinfluences a current state of the lighting control sub-system, or thereceived information includes parameters that influence a future stateof the lighting control sub-system. For an embodiment, the receivedinformation influences a lighting control sub-system profile. For anembodiment, the lighting sub-system profile includes a set of values(parameters) that affect the operation of the controller in determininghow it controls the light output of the luminaire based on current andpast sensor inputs, time of day or passage of time. For at least someembodiments, the parameters are adaptively updated.

For at least some embodiments, the controller is operative to receive aplurality of lighting control sub-system profiles. That is, there can bemore than one lighting control sub-system profile, and the lightingcontrol sub-system profiles can be adaptively updated. Morespecifically, an active profile or present profile of the plurality oflighting control sub-system profiles can be adaptively updated. Further,for at least some embodiments, the external device can add, replace ordelete one or more profiles of the plurality of lighting controlsub-system profiles.

As previously stated, the external device can be a central controller oranother lighting control sub-system. Further, the external device caninclude a logical group controller, or a terminal. For at least someembodiments, a terminal is a device that allows interaction with thelighting control sub-system in a form that can be done by humans in alanguage that is human readable.

An embodiment of a logical group controller provides control over aplurality of lighting control sub-systems, wherein the plurality oflighting control sub-systems are all a member of the logical group. Thesize of a logical group is dynamic. Further, any one of the lightingcontrol sub-systems can be member of any number of logical groups,wherein each logical group can include a different set of members. Foran embodiment, the external device looks like a traditional light wallswitch but with several buttons or controls. The external device is usedto affect all lighting control sub-systems in the logical group to turnthem on or off, and/or to set them to configured light levels that areappropriate for the use of the space in which the logical group oflighting control sub-systems is located. For example, such as viewingoutput from a projector on a screen, or listening to a formalpresentation.

At least some embodiments include a plurality of sensors, wherein thecontroller is operative to control the light output based on acombination of sensed inputs of the plurality of sensors. The sensorscan include, for example, ambient light sensors and occupancy sensors.Additionally, timing and scheduling controls can be provided by clocksand/or timers. At least some embodiments further include the controlleroperative to control the light output based on the combination of one ormore of the sensed inputs and a lighting schedule. For example, during awork time (known occupancy of, for example, an office building) a lightin a work area may be restricted from turning off. However, duringnon-work times, the light can be turned off if no one is present. Ifutilizing a lighting schedule, clearly the lighting control sub-systemincludes and/or has access to a clock and/or a timer.

For at least some embodiments, the controller is operative to receive alighting control configuration. For an embodiment, the lighting controlconfiguration includes the above-described lighting sub-system profile.For an embodiment, the controller receives the lighting controlconfiguration from a system operator. This includes (but is not limitedto), for example, a situation where an operator sets the configurationusing dip switches that are physically located on the sub-system. For anembodiment, the controller receives the lighting control configurationfrom a central controller, thereby allowing a system user to manage thelighting control configuration.

For an embodiment, the controller is operative to collect sensor valuesover time based on at least the sensed input. Again, the controller hasaccess to a clock and/or a timer. For an embodiment, the controller isoperative to communicate the collected sensor values to the externaldevice. For example, a value of occupancy can be determined every Xseconds, saved for the last Y minutes, and then reported to the externaldevice or central controller. For an embodiment, the controller isoperative to identify problems of operation of the lighting controlsub-system based on the collected sensor values, and to report theidentified problems of operation to the external device. For example,the controller can report that the temperature is too high or too low.The controller can report that a light has burned out, or report a lossof coupling with the luminaire. For at least some embodiments, thecontroller is operative to report past operating characteristics of thesub-system. For example, the controller can report light level changes.

For at least some embodiments, the sensor includes a power monitoringunit (such as, the energy meter 322) operative to measure power usage ofthe lighting control sub-system. Further, for at least some embodiments,the controller is operative to communicate the measured power usage ofthe sub-system to the external device.

As will be described in detail, for at least some embodiments, thecontroller is operative to communicate with other sub-systems, andidentify logical groups of two or more sub-systems. For at least someembodiments, identifying logical groups comprises at least thecontroller and at least one of the other sub-systems auto-determiningthe logical group. For an embodiment, at least one of the logical groupsincludes a motion sensing group. For an embodiment, at least one of thelogical groups includes an ambient light group. For an embodiment, atleast one of the logical groups includes a logical switch group. For anembodiment, at least one of the logical groups includes a logicaltemperature group.

For an embodiment, the controller is operative to control the lightoutput based on a sensor signal of a sensor of another sub-system of acommon logical group. For an embodiment, sub-systems of a common logicalgroup communicate to each other when a sensor of one of the sub-systemsof the logical group has failed.

For an embodiment, the controller is operative to identify an emergencycondition, and initiate an emergency mode. For a specific embodiment,the controller is operative to confirm the identification of theemergency mode, including the controller initiating communication with anon-emergency device, and confirming the identified emergency conditionif the initiated communication is not successful.

FIG. 4 shows a smart sensor system 302 and an infrared (IR) illuminator490, according to an embodiment. For at least some embodiments, the IRilluminator 490 is powered if lighting in the area of interest is sensedto be below a lighting threshold. Specifically, for an embodiment, theoccupancy detection includes analyzing a plurality of images acquired ata location in which occupancy is being detected, and powering the IRilluminator 490 proximate to the image sensor 344 that is obtaining aplurality of images if lighting proximate to the image sensor 344 issensed to be below a lighting threshold.

As shown, for this embodiment, the IR illuminator 490 includes acontroller 492, and an input that allows for interrupted operation ofthe IR illuminator 490. As shown, for an embodiment, the IR illuminator490 generates IR light that is reflected off of objects of the area ofinterest. The image sensor 344 senses images based at least in part onthe reflected IR light.

For an embodiment, the IR illuminator 490 is placed adjacent to theimage sensor 344 and/or the smart sensor system 302. For an embodiment,the IR illuminator 490 is integrated with the image sensor 344 and/orthe smart sensor system 302.

At least some embodiments include or allow for reducing (adjusting) aframe or image rate of the plurality of images. Further, at least someembodiments include the IR illuminator 492 being enabled and disabled sothat its on-periods coincide only with the required exposure time ofimage capture by the image sensor. In between image captures by theimage sensor, the IR Illuminator is disabled. Further, at least someembodiments include adapting the rate of frame capture by the imagesensor, together with the synchronous activation of the infraredilluminator, based on a desired or require power consumption.

FIG. 5 is a flow chart that includes steps of a method of determiningoccupancy, according to an embodiment. A first step 510 includesobtaining a plurality of images. A second step 520 includes determininga color difference image of a color difference between consecutiveimages of the plurality of images. A third step 530 includes determiningan area of the color difference image wherein the color difference isgreater than a threshold. A fourth step 540 includes creating anoccupancy candidate based on filtering of the determined area if thedetermined area is less than light change threshold. For an embodiment,this includes applying a Gaussian, or Gaussian-like, filter to thethresholded difference image and threshold the filtered result. Anembodiment includes applying a connected component analysis to the imageobtained to assign a unique label to each connected component. Eachconnected component represents an occupancy candidate region. A fifthstep 550 includes generating one or more centroids that represent areasof the occupancy candidate that are greater than a second threshold.Stated differently, this step includes generating one or more centroidsof the areas that represent the occupancy candidate regions. A sixthstep 560 includes tracking the one or more centroids over a period oftime. A seventh step 570 includes detecting occupancy if one or more ofthe centroids is detected to have moved more than a movement thresholdover the period of time.

For an embodiment, the steps of determining a color difference image ofa color difference between consecutive images, and determining an areaof the color difference image wherein the color difference is greaterthan a threshold includes converting the plurality of images to a greyscale, determining an absolute value of a difference in the grey scalebetween consecutive images, and determining an area of the plurality ofimages wherein the absolute value is greater than a threshold.

More generally, an embodiment of the step of determining a colordifference image of a color difference between consecutive imagesincludes determining the difference between pixel values in successiveimages. For at least some embodiments, the difference is computedbetween color values of pixels, or between grey values of pixels. If thedifference is computed between color values a conversion to L*a*b* orL*C*h color spaces may be required.

As previously described, an embodiment further includes down-samplingthe plurality of images before converting the plurality of images to agrey scale. For an embodiment, converting the plurality of images to agrey scale includes processing a green channel.

For an embodiment, if the determined area is greater than light changethreshold, then it is determined that a lighting changes has occurrednot detection of occupancy.

As previously described, for an embodiment filtering of the determinedarea includes summation filtering of the determined area.

As previously described, an embodiment further includes powering aninfrared (IR) illuminator proximate to a device that is obtaining aplurality of images if lighting proximate to the device is sensed to bebelow a lighting threshold. An embodiment includes reducing a frame orimage rate of the plurality of images. Further, for an embodiment the IRilluminator is strobed with a pattern that is synchronous with the framerate of the plurality of images. The rate can be decreased or controlledto reduce power consumption.

As previously described, an embodiment further includes measuringtemporal sequences of movement of the centroids, and comparing themeasured temporal sequences to a database of temporal sequences known tobe associated with humans, allowing an estimate of a likelihood that oneor more humans appear in the plurality of images.

FIG. 6 shows a lighting control sub-system, according to anotherembodiment. For at least some embodiments, the lighting controlsub-system of FIG. 6 is operationally similar to that of FIG. 3. Incontrast to FIG. 3, the embodiment shown in FIG. 6 includes a singlecontroller 620 which controls an intensity of emitted light from aluminaire 640 based on sensed signals from, for example, one or more ofa light sensor 631, a motion sensor 632, an temperature sensor 633, acamera (or other type of image sensor) 634 and/or a air quality sensor635. Clearly, other types of control other than control of a luminairecan additionally or alternatively be included. Further, additional oralternate sensor types can be included.

FIG. 7 shows a room wherein the reflectivity of objects within the roomis characterized, according to an embodiment. FIG. 7 shows a room(merely as an example, clearly, other types of areas of interest couldbe depicted) that includes an area of interest in which ambient lightlevels are to be detected. As shown, a luminaire 712 emits light. Theemitted light reflects off of luminance target areas 750, 760, and thereflected light is received by an image sensor 710. While only a singleimage sensor 710 is shown, clearly additional images sensors can beincluded. As shown, the image sensor maintains a communication link witha server/controller 720. As previously described, the server/controller720 may be included within the image sensor 710, or theserver/controller 720 may be a separate unit. Additionally, theserver/controller 720 can be networked (for example, through the cloud)to external servers and/or controllers.

For at least some embodiments, the server/controller 720 and/or anexternal controller electronically connected to the controller areoperative to analyze the plurality of images acquired (by the imagesensor 710) at a location (the room or area of interest 700) in whichoccupancy is being detected. Further, the server/controller 720 and/oran external controller electronically connected to the controller areoperative to select at least one target area (such as, target areas 750,760) within the plurality of images which is best suited as an targetarea for ambient light level sensing, calculate a reflectivity estimatefor each of the at least one target area using knowledge of values ofapplied lighting (for example, values of the lighting intensity of theluminaire 712), calculate ambient lighting estimates for the at leastone target area using the calculated reflectivity estimates for the atleast one target area, and continuously calculate a total light level inthe area of interest using the ambient lighting estimates for the atleast one the target.

Descriptively, the described embodiments perform a calibration thatincludes picking a target geometrical area, measuring a luminance of thetarget geometrical area using a controlled change to the light level tocalculate the reflectivity of the area. The controlled change to thelighting level can be implemented by, for example, switching onindividual lights at night when there is little or noexternally-generated ambient light. For example, the controlled changeof the lighting can be performed, for example, at night when the ambientlight passing through, for example, the window 750 is low. Thecalibration can be repeated for a number of areas in the image.

For at least some embodiments, the calibration results in a vector ofsampled image areas and associated reflectivity estimates. For at leastsome embodiments, in operation, the reflectivity estimates are used tocalculate an overall ambient light estimate for the area of interest. Ifone particular target area varies by more than a threshold above whatthe whole scene brightness varies, then it is determined that area hasundergone a reflectivity change and that area can be marked forrecalibration.

The described embodiments can be used with a single area covering thewhole area of interest in which case the aggregate reflectivity of thewhole area of interest is calibrated and the aggregate reflected lightused to estimate the ambient light level. A calibrated source of lightis required to make the reflectivity estimates. This can be obtained bymodeling a luminaire and bulb/tube combination as long the luminaire isunder the control of the controller. The modeling can consist of eithermeasuring the output of the luminaire, or the light generated inside theluminaire, using a brightness sensor and combining that with a model ofthe luminaire optics and area of interest geometry to generate anestimate of the calibrator light falling on the area of interest, ormeasuring electrical characteristics of the bulb/tube, and combiningthat with a model of the bulb/tube and the luminaire and the area ofinterest geometry to generate an estimate of the calibrator lightfalling on the area of interest, or use a calibrated strobe light (whichmay be low intensity) to illuminate the area of interest.

FIG. 8 is a flow chart that includes steps of another method ofdetecting occupancy, according to an embodiment. A first step 810includes obtaining a plurality of images. A second step 820 includesdetermining a color difference image of a color difference betweenconsecutive images. For at least some embodiments, the color differenceimage is computed by calculating the absolute value of the distance incolor space between corresponding pixels in consecutive images. Examplesof suitable color spaces are CIE76, CIE94 and CIEDE2000. To calculatethe color difference image, the consecutive images from the image sensormust first be converted into the selected color space. A third step 830includes determining areas of the color difference image wherein thecolor difference is greater than a threshold. For an embodiment, theabsolute value of the difference between a present frame and theprevious one is computed and threshold detected at some level toeliminate noise. Any differences that exceed the threshold level areassumed to be the result of something moving in the interval between theframes. A fourth step 840 includes calculating a total change area as anaggregate area of the determined areas. A fifth step 850 includescreating a list of occupancy candidate regions based on filtering of thedetermined areas if the total change area is less than a light changethreshold, wherein each occupancy candidate region is represented by aconnected component. For an embodiment, the light change threshold isselected to be the minimum number of changed pixels that represent achange in lighting conditions. If more pixels have changed than thelight change threshold, a lighting change is deemed to have occurred.For an embodiment, the light change threshold is a function of thedistance from the image sensor to the area of interest, the size ofmoving objects in the area of interest, and the optical properties ofthe background of the area of interest and any moving objects in thearea of interest. A sixth step 860 includes tracking the one or moreoccupancy candidate regions over a period of time. A seventh step 870includes detecting occupancy if one or more of the occupancy candidateregions is detected to have moved more than a movement threshold overthe period of time.

As previously described, an embodiment further includes powering aninfrared (IR) illuminator proximate to a device that is obtaining aplurality of images if lighting proximate to the device is sensed to bebelow a lighting threshold. As previously described, an embodimentfurther includes reducing a frame or image rate of the plurality ofimages. As previously described, for an embodiment the IR illuminator isstrobed with a pattern that is synchronous with a frame capture timessequence of the plurality of images.

As previously described, an embodiment further includes measuringtemporal sequences of movement of the centroids, and comparing themeasured temporal sequences to a database of temporal sequences known tobe associated with humans, allowing an estimate of a likelihood that oneor more humans appear in the plurality of images.

Although specific embodiments have been described and illustrated, thedescribed embodiments are not to be limited to the specific forms orarrangements of parts so described and illustrated. The embodiments arelimited only by the appended claims.

What is claimed:
 1. An apparatus, comprising: at least one image sensor;a controller, wherein at least one of the controller, or an externalcontroller electronically connected to the controller are operative to:obtain a plurality of images from the at least one image sensor;determine a color difference image of a color difference betweenconsecutive images of the plurality of images; determining an area ofthe color difference image wherein the color difference is greater thana threshold; create an occupancy candidate based on filtering of thedetermined area if the determined area is less than a light changethreshold; generate one or more centroids that represent areas of theoccupancy candidate that are greater than a second threshold; track theone or more centroids over a period of time; and detect occupancy if oneor more of the centroids is detected to have moved more than a movementthreshold over the period of time.
 2. The apparatus of claim 1, whereindetermining a color difference image of a color difference betweenconsecutive images, and determining an area of the color differenceimage wherein the color difference is greater than a threshold comprisesthe at least one of the controller, or the external controllerelectronically connected to the controller are operative to: convert theplurality of images to a grey scale; determine an absolute value of adifference in the grey scale between consecutive images; and determinean area of the plurality of images wherein the absolute value is greaterthan a threshold.
 3. The apparatus of claim 1, wherein at least one ofthe controller, or the external controller are further operative tocontrol at least one environmental parameter of an area of interestbased on the detected occupancy.
 4. The apparatus of claim 2, wherein atleast one of the controller, or an external controller electronicallyconnected to the controller are further operative to down-sample theplurality of images before converting the plurality of images to a greyscale.
 5. The apparatus of claim 2, wherein converting the plurality ofimages to a grey scale comprises processing a particular color channel.6. The apparatus of claim 1, wherein if the determined area is greaterthan the light change threshold, then determining that a lighting changehas occurred, not a detection of occupancy.
 7. The apparatus of claim 1,wherein filtering of the determined area comprises summation filteringof the determined area.
 8. The apparatus of claim 1, further comprisingpowering an infrared (IR) illuminator proximate to the at least oneimage senor of the apparatus if lighting proximate to the at least oneimage sensor is sensed to be below a lighting threshold.
 9. Theapparatus of claim 8, further comprising reducing a frame rate of theplurality of images.
 10. The apparatus of claim 9, wherein the IRilluminator is operative to strobe with a pattern that is synchronouswith a frame capture times sequence of the plurality of images.
 11. Theapparatus of claim 1, wherein at least one of the controller, or anexternal controller electronically connected to the controller arefurther operative to measure temporal sequences of movement of thecentroids, and compare the measured temporal sequences to a database oftemporal sequences known to be associated with humans, allowing anestimate of a likelihood that one or more humans appear in the pluralityof images.
 12. The apparatus of claim 1, wherein at least one of thecontroller, or an external controller electronically connected to thecontroller are further operative to: analyze the plurality of imagesacquired of an area of interest in which occupancy is being detected;select at least a target area within the plurality of images which isbest suited as an ambient target area for ambient light level sensing;calculate a reflectivity estimate for each of at least one target areausing knowledge of values of applied lighting; calculate ambientlighting estimates for the at least one target area using the calculatedreflectivity estimates for the at least one target area; andcontinuously calculate a total light level in the area of interest usingthe ambient lighting estimates of at least one target area from theplurality of images.
 13. A method of detecting motion, comprising:obtaining a plurality of images; determining a color difference image ofa color difference between consecutive images of the plurality ofimages; determining an area of the color difference image wherein thecolor difference is greater than a threshold; creating an occupancycandidate based on filtering of the determined area if the determinedarea is less than light change threshold; generating one or morecentroids that represent areas of the occupancy candidate that aregreater than a second threshold; tracking the one or more centroids overa period of time; and detecting occupancy if one or more of thecentroids is detected to have moved more than a movement threshold overthe period of time.
 14. The method of claim 13, wherein determining acolor difference image of a color difference between consecutive images,and determining an area of the color difference image wherein the colordifference is greater than a threshold comprises converting theplurality of images to a grey scale; determining an absolute value of adifference in the grey scale between consecutive images; determining anarea of the plurality of images wherein the absolute value is greaterthan a threshold.
 15. The method of claim 14, further comprisingdown-sampling the plurality of images before converting the plurality ofimages to a grey scale.
 16. The method of claim 14, wherein convertingthe plurality of images to a grey scale comprises processing aparticular color channel.
 17. The method of claim 13, wherein if thedetermined area is greater than light change threshold, then determiningthat a lighting changes has occurred not detection of occupancy.
 18. Themethod of claim 13, wherein filtering of the determined area comprisessummation filtering of the determined area.
 19. The method of claim 13,further comprising powering an infrared (IR) illuminator proximate to adevice that is obtaining a plurality of images if lighting proximate tothe device is sensed to be below a lighting threshold.
 20. The method ofclaim 19, further comprising reducing a frame or image rate of theplurality of images.
 21. The method of claim 20, wherein the IRilluminator is strobed with a pattern that is synchronous with a framecapture times sequence of the plurality of images.
 22. The method ofclaim 14, further comprising measuring temporal sequences of movement ofthe centroids, and comparing the measured temporal sequences to adatabase of temporal sequences known to be associated with humans,allowing an estimate of a likelihood that one or more humans appear inthe plurality of images.
 23. The method of claim 13, further comprising:analyzing the plurality of images acquired of an area of interest inwhich occupancy is being detected; selecting at least a target areawithin the plurality of images which is best suited as an ambient targetarea for ambient light level sensing; calculating a reflectivityestimate for each of at least one target area using knowledge of valuesof applied lighting; calculating ambient lighting estimates for the atleast one target area using the calculated reflectivity estimates forthe at least one target area; and continuously calculating a total lightlevel in the area of interest using the ambient lighting estimates of atleast one target area from the plurality of images.